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Abstract 

o 

This paper deals with the regularity of solutions of the Hamilton- Jacobi Inequality which 
arises in Hoc control. It shows by explicit counterexamples that there are gaps between 
existence of continuous and locally Lipschitz (positive definite and proper) solutions, and 
between Lipschitz and continuously differentiable ones. On the other hand, it is shown that 
^5 it is always possible to smooth-out solutions, provided that an infinitesimal increase in gain 

^3 . is allowed. 

Keywords: control, storage functions, dissipation inequalities, Lyapunov functions, sta- 
bility, viscosity solutions 
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1 Introduction 

The so-called "i^oo control problem" is that of finding a state (or more generally, a measurement- 
based) feedback which stabilizes a given system, while satisfying an energy-gain (L 2 operator 
norm) constraint. For linear systems, the problem has a long history, and an elegant solution 
was provided in the by now classical paper [||. The nonlinear version of this problem has been 
the subject of intense research as well; see for instance ||, 13, 20] among many others. A 



central role in these studies is played by a partial differential inequality, a Hamilton-Jacobi 
Inequality (HJI) which is satisfied by a "storage" or "energy" function V associated with the 
closed loop system. 

In this paper, we concern ourselves with the analysis of the HJI for a system already in 
closed-loop form, since we wish to make some remarks about the regularity of solutions of this 
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equation. Moreover, in order to keep the discussion as simple as possible, we analyze the case 
of full state measurements, but similar conclusions could be drawn for the case when outputs 
are of interest. Thus, the main focus of this paper will be on systems, affine in inputs, of the 
following form: 

m 

x = g {x) + ^2 u i9i{x) (1) 

i=l 

for which states x evolve in R n , and inputs u = (u±, . . . , u m ) take values in R m . We assume that 
the vector fields i = 0, . . . ,m are locally Lipschitz in x. (These systems might be thought 
of having been obtained from a more general class of systems x = f(x,u,v) after applying a 
stabilizing feedback v = k(x), but that interpretation is irrelevant to the results that we give.) 
In this introduction, we restrict attention to the above class of systems, but later we will also 
provide several results valid for more general systems, not necessarily affine in inputs, of the 
form 

x = F(x,u), (2) 

with x(t) G M. n and u(t) 6 IA. (As far as the definitions are concerned, we do not need impose 
any technical conditions on IA and F.) We use |-| to denote Euclidean norm in 1" or W n . For 
systems ([!]), the HJI's of interest are inequalities which are often expressed as: 

. m 

W(x) • go(x) + — (yV(x) ■ 9i (x)) 2 + \x\ 2 < , (3) 
^ i=i 

where 7 > is the (square of the) "L 2 gain" of the system. Inequality (E3) for non-differentiable 
V must be interpreted in a generalized sense, as we discuss below. The V's that arise, because 
of implicit stability considerations, must be proper and positive definite. 

Modulo some elementary calculus to do the implied maximization over u, the inequality (|3|) 
is equivalent to: 

W(x)- ^go(x) + Y^u l9i {x)^ < -|x| 2 +7M 2 , (4) 

understood as holding for all u. This latter form is preferred, because it makes sense for 
arbitrary (not necessarily input-affine) systems (0) , and because the theory then makes a natural 
contact with the theory of dissipative systems developed by Willems and Moylan and Hill (see 
e.g. [|ll], 12, |2(], ^T|). Under suitable technical assumptions, one can show that (||) is equivalent 



to the following inequality: 

f b 

V(x(b))-V(x{a)) < / \u{t)\ 2 - \x{t)\ 2 dt (5) 

J a 

holding along all solutions (x(-), u{-)) of (jlj) with x(-) absolutely continuous and u(-) measurable 
and essentially bounded, for each a < b in the domain of the solution. This follows from general 
results about proximal characterizations of "strong invariance" of differential inclusions given 



in Chapter 4 of ||, or the equivalent viscosity characterizations given e.g. in [14], see also the 
discussion in S. Notice that @ has the following consequence for trajectories which start at 
x(0) = 0, and if V is nonnegative: for all T > for which the solution is defined, it must hold 
that 

rb 

\x(t)\ 2 dt < 7 / j\u(t)\ 2 dt. 
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This means that the map u(-) t— > x(-) seen as a map between square- integr able functions, has 
operator norm < ^Fj. Conversely, the theory relates such operator norms back to the existence 
of V's solving the differential inequality. We will not discuss this well-known material any 
further, but rather consider the differential inequality as our object of study. 

In general, it is not natural to impose the requirement that solutions to (|J) should be 
smooth, so the HJI must be interpreted in the viscosity sense (a "viscosity supersolution"), see 
e.g. j?], 0]), or, in what is an essentially equivalent manner, the proximal analysis formalism 
found in Under suitable controllability hypotheses, it does make sense to restrict attention 
to continuous V's, see for instance ||. Thus we will always assume in our study that V is at 
least continuous. 

The main results in this note address the gap between continuous and C 1 solvability. We give 
examples which show (a) that there may exist even Lipschitz-continuous (proper and positive 
definite) solutions, but no possible continuously differentiable ones, and (b) there may exist 
continuous (proper and positive definite) solutions but no possible locally Lipschitz continuous 
ones. On the other hand, we provide a smoothing result which shows that it is possible to pass 
from C° to C 1 solutions as long as an infinitesimal increase in the gain 7 is allowed. A last 
section treats the special case of one-dimensional systems, where no gap exists. None of these 
facts is unexpected, of course, but it would appear that the corresponding counterexamples and 
proofs are not available in the literature. 



2 Definitions and Statements of Results 



Recall (cf. ||, Section 3.4) that a vector £ € R n is a viscosity subgradient of a function V : 
R n -» R, at the point x G W 1 , if 

liminf-r— [V(x + h)~ V{x)-C,-h] > (6) 
h— >o \h\ 

The (possibly empty) set of all viscosity subgradients of V at x is called the viscosity subdif- 
ferential, and is denoted c\>V{x). Observe that, if the function V is differentiable at x, then 
chV(x) = {VV(x)}. 

Definition 2.1 Suppose given a system E as in (||), and a 7 > 0. We say that a function 
V : R n — * R>o witnesses the gain 7 if the following condition holds: 

(■F(x,u) < -|x| 2 + 7|n| 2 VCed D V(x) (7) 

for all x 6 W 1 \ {0} and u £U. The set of all those continuous V which witness a given gain 7 
for the system E is denoted as W(E,7). □ 



We recall that a continuous function V : M. n — > R>o is said to be proper (or "radially 
unbounded") provided that every set of the form {x\ V(x) < a} is compact, for each a > 
(equivalently, V(x) — ► 00 as \x\ — > 00), and is said to be positive definite if V(x) = if and only 
if x = 0. We denote 

Woo(E,7) := {V £ W(E,7) | V is proper and positive definite } . 

When V is C 1 , condition (Q) means simply that VV(x) • F(x,u) < — \x\ 2 + 7|n| 2 for all 
1 / and u. As is well-known, asking for a globally C 1 such function which is also positive 
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definite is overly restrictive. To see this, consider an input-affine system with n = 1, and take 
u = 0. The inequality would force the bound \x\ 2 / \g$(x)\ < \V'(x)\ for all x ^ 0. If V is 
positive definite, it must have a local minimum at zero, so V'(0) = 0. Hence 

\x\ 2 = o(go(x)) as x — > , 

which is too strong a constraint on go (one would not be able to study a system such as x = —x 3 ). 
Therefore, it is routine (see for instance |19|J) to drop the requirement of differentiability at the 
origin. We will denote by Cq the set of continuous functions V : W 1 — > M>o whose restriction 
to W 1 \ {0} is continuously differentiable. 

The first result consists of an example which shows that there may exist a globally Lipschitz 
V 6 Woo(S, 7), but no Cq such function (in fact, not even merely a Cq proper, nor a Cq positive 
definite, function in W(S,7)). Specifically, take the following system Si: 

x\ = \x\\ (— x-y + \x%\ + til) 
±2 = x 2 (-xi - \x 2 \ + u 2 ) ■ 

Note that this system is of the form (|l]), with n = m = 2. Now consider the following function 
(basically, the L 1 norm): 

V x (x) = 2|xi| + 2|x 2 | . 
This function is proper and positive definite, and globally Lipschitz. We prove in Section ||: 

Proposition 2.2 For the above system Si, and unit gain 7 = 1, we have V\ G Woo (Si, 1). On 
the other hand, if V is any Cq function in W(Si, 1), then V is not positive definite nor proper. 

Remark 2.3 We chose to give example Si because of its simplicity. However, at the cost of 
added complexity, one may easily provide similar examples which are such that the vector fields 
gi are not merely Lipschitz. For a C 1 example, we may take 

±1 = |xirr 2 | 3 + x\ {—\x\\ + u{) 
x 2 = -(xix 2 ) 3 + x 2 (-\x 2 \ + u 2 ) ■ 

The proof of the analog of Proposition |2.2| , using the same function V\ and gain 1, is virtually 
the same. □ 

The second result produces an example showing that there may exist a continuous V G 
Woo(S,7), but every locally Lipschitz function in W(S,7) must be either nonproper or non- 
positive definite. Specifically, we will consider the following system S3: 

Xl = —Xl + X 2 + U\ 

x 2 = Sxt/^ (-Xl - x 2 + u 2 ) 
which is of the form ([!]), with n = m = 2. We will consider the following function: 

V3(xi,x 2 ) := x\ + x 2 2 /3 
which is proper, positive definite, and continuous. We prove in Section ||: 
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Proposition 2.4 For the above system S3, and unit gain 7 = 1, we have V3 E Woo (S3, 1). 
On the other hand, if V is any locally Lipschitz function in W(S3, 1), then V is not positive 
definite nor proper. 

There is a general positive result as well. We show that, for any system S as in (|l|), and 
any 7 > 0, Woo(S, 7 ) 7^ implies that Woo(S,7')nCo 7^ for each 7' > 7. In other words, 
it is always possible to smoothly approximate a proper positive definite continuous V by one 
that is continuously differentiable away from zero (actually, the proof provides an infinitely 
differentiable such approximation), provided that we allow a negligible increase in gain. This 
is summarized in the following statement, which we also prove in Section |3|: 

Theorem 1 For any system S as in (Q), 

inf{ 7 I W oo (S, 7 )/0} = inf{ 7 I Woo(S, 7 )f|C o V0} • 

This result is significant in so far as the "inf" in question is the one of interest in control 
problems. 

Remark 2.5 The result in Theorem [l] is stated for input-affine systems. It is false in general 
for arbitrary systems (||). One elegant statement can be made by considering systems of the 
following special form, for any p > 1: 

771 

x = go{x) + ^2 uP i9i{x) (8) 

i=l 

(again assuming that the vector fields gi, i = 0, . . . ,m are locally Lipschitz in x). We may 
interpret the powers u p , for negative u, either as or as "signu |it| p "; with either interpre- 

tation, we shall show, also in Section ||, that Theorem [l] holds (for any system of this form) if 
p < 2 and does not hold (for some systems of this form) if p > 2. □ 

Finally, we will analyze in Section |] the special case m = n = 1. We will show that there is 
in that case no gap between the locally Lipschitz and the differentiable case, for systems affine 
in inputs, but that a gap reappears if we deal with systems that are not input-affine. 



3 Proofs of Main Results 

We first prove Proposition and Proposition [2.4| . We then prove Theorem |l| (in somewhat 
more generality), and, finally, we justify the claims made in Remark 2.5. 



3.1 Proof of Proposition 2.2 



In order to verify that V\ E W(Si, 1), we compute its subgradients. At any point x = (x\, X2) E 
M 2 with X1X2 7^ 0, obviously 



3dVi(x) = {Wi(x)} = \2[p- v p^ 
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so / becomes 



Wi(a;)- \ g {x) +y^Uigi(x) } = 2-r— \x x \ (-xi + |x 2 | + u{) + 2—^-x 2 (-xi - |x 2 | + u 2 ) 

» / Fl F2 



= 2tiiXi + 2n2|x2| — 2x\ — 2x2 

^ I |2 I 1 2 

< |u| — |x| , 

as desired. Suppose now that x\ = and X2 7^ 0. Then, it is an easy exercise with the definition 
of subgradients to see that 

X2 



doVi(x) = [-2,2] x ^2- 

l F2| 

so in this case, for any ( = (£1, £2) £ ^D^l(^)> we have: 



C • ^go(^) + ^#0*0 J = Ci • + 2|-^|X 2 (-|x 2 | + u 2 ) 

1 1 9 

= 2u2p2| — 2x 2 

< |^2 1 — \X2\ < M — \X\ , 

again as desired. (The actual form of Ci turns out to be irrelevant.) The case (x\ 7^ 0, X2 = 0) 
is similar. So V\ G W(Ei,l). 

We next show that, if E W(Si, 1) is of class Cq, then it cannot be proper nor positive 
definite. So, assume that 

V^)| ft (i)l^,ftWj < |n| 2 -|x| 2 (9) 

holds for all x 7^ and all u. Fix any number a > 0. Consider any point of the form 
x = (xi,X2) = (a, X2), with X2 7^ 0. With the special choice u\ = x\ and u 2 = \x 2 \, we have 
\x\ = \u\, so (using subscripts to denote partial derivatives) inequality @ reduces to: 

W Xl (a,x 2 ) - (signx 2 )W X2 (a,x 2 ) < 0. 

When X2 > this gives W Xl (a, x 2 ) — W X2 (a, x 2 ) < 0, so taking the limit as X2 — > + , we conclude 

W Xl (a,0)-W X2 (a,0) < 
for all a > 0. Arguing with negative X2 and taking X2 — > 0~, we get also 

W« 1 (o,0) + W ai3 (a,0) < 

for all a > 0. We conclude that W Xl (a,0) < for all a > 0. This means that W(-,0) must be 
bounded above (and is nonnegative on R>o), so W cannot be proper. On the other hand, if 
W(0) = 0, this implies W(a, 0) = 0, so W cannot be positive definite either. This completes 
the proof of Proposition |2.2|. 
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3.2 Proof of Proposition 



2.4 



The intuitive idea of the construction of X2 and V2 is as follows. We start with Vi{x\,x<i) = 
x\ +x?,, the square norm in M 2 , (xi,X2) denoting the canonical coordinates on M 2 , and consider 
the harmonic oscillator motion, but with rescaled time so that the xi-axis consists of equilibria: 
g(x\,X2) = (X2X 2 ., ~~ X1X2). Note that V2 is an integral for the motions of g. Next, we make 
a change of coordinates which is a global homeomorphism but fails to be a diffeomorphism: 
x\ 1 — > x\ =: x\,X2 1— > &2 =: X2- In the new coordinates (xi,X2), V2, the transformation of V2, is 
no more locally Lipschitz. However, the transformation g of g is locally Lipschitz, and it is of 
course still true that V2 integrates g. The key fact is this: for any locally Lipschitz W which 
is nonincreasing along trajectories of g, W cannot be positive definite nor proper. (In other 
words, there cannot be any locally Lipschitz "weak Lyapunov function" for x = g(x)). Finally, 
we add an input-dependent term to g which provides the dissipation property. When \u\ = \x\, 
this dissipation property reduces to the nonincreasing property mentioned above, and hence 
leads to a contradiction for V locally Lipschitz. 

We define the following locally Lipschitz vector field g in M 2 : 

5(X1 ' X2) = (-3x?xf ) 

and note that: 

C-g(x) = V(ed D V 2 (x), V1GR 2 . (10) 

— 1/3 

Indeed, pick any x = (3:1,2:2). If X2 7^ then necessarily ( = (2a;i, (2/3)x 2 ), so the claim is 
clear. If instead X2 = 0, then g(x±, X2) = 0, so the claim holds as well. 

Note that system E2 is of the form x = f(x, u), where f(x, u) = g(x) + h(x, u), with 

h(x,u) := ( „ 4/3 1 / Xl \ J ■ 

\^X 2 ' {U2 ~ X 2 ) J 

Now note that: 

(■ h(x,u) < \u\ 2 -\x\ 2 V(ed D V 2 (x), Vx G M 2 ,Vu G M 2 . (11) 

To verify this, pick any x = (2:1,2:2). Take first the case X2 7^ 0. Since ( must be equal to 
(2xi, (2/3)x 2 1 ^ 3 ), we have that 

( ■ h(x, u) = 2x\U\ — 2x 2 + IX2U2 — 2x2 — \ u \ 2 ~ \ x \ 2 ■ 

If instead X2 = 0, then C = (2xi, C2) , for some £21 so 

C • h{x, u) = 2x\U\ — 2x\ + C2 • < u\ — x\ < \u\ 2 — |x| 2 . 

Together with (^0|), we have thus that £ ■ f(x,u) < \u\ 2 — |x| 2 , so we have proved that V2 G 
Woo (S2, 1). Now we show that we cannot have a locally Lipschitz V G Wooi^, !)• 

For any V G W(S2, 1), setting u = x in the inequality £ • f(x,u) < \u\ 2 — |x| 2 , and using 
that h(x, x) = 0, gives that 

(■g{x) < VC G dvV{x) , Vx G R 2 . (12) 

We will show that no locally Lipschitz function V can satisfy such a property and be positive 
definite or proper. Suppose given such a V; we will prove that, for any positive number a at 
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which the locally Lipschitz function x\ i— > V{x\,$) is differ entiable, necessarily V Xl (a, 0) < 0. 
This means that V decreases along the xi-axis, and the negative conclusion follows. Fix now 
any such a, and denote £ := (a, 0)'. We will analyze the behavior of V in a neighborhood of £, 
by considering a curve which approaches £ along an orbit of g. 

Take the following parameterized curve: 

7:[0,l]-M 2 :t-( (a2 _ ( ^ )2)3/2 ) 
which has the property that 7(1) = 7'(1) = £. Note also that, for each t G [0, 1], 

where /9(t) = \{a 2 - (at) 2 ) 3 / 2 > for all t < 1. 

Lemma 3.1 The function i G [0, 1] 1— ► W := V(7(t)) is nonincreasing. 

Proof. Since W is locally Lipschitz, its derivative exists almost everywhere. We must prove 
that W'(t) < for almost all t. This will follow from the following statement, valid for all 
subgradients: 

77 < for all t e [0, 1) and all rj G d D W{t ) . (13) 
The idea of the proof is as follows. If W(7(to)) exists, then 

W'(t ) = W( 7 (to))Y(*o) = _L-W( 7 (to))0(7(*o)) < 

/?(*o) 



where the last inequality follows from (12). However, there is no reason for V to be differen- 



tiable at the points in the image of 7. So we need to apply the "approximate chain rule" for 
subgradients. Pick any to G [0, 1) and rj G flt>W(io). Take any e > 0, and let < 5 < e be such 
that 

Id \g(x) - g( 7 (t ))\ < s/3(t ) (14) 

for all C G di)V(x) and all x such that \x — 7(^0) I < ^ (using continuity of 5, and noting that 
all such £ are bounded by a Lipschitz constant for 1/ on a ball of radius 5 about 7 (to)) an d also 
so that 

ICI |Y(<i)-Y(*o)| < e (15) 
whenever £ G doV^x) for any x as above and |ti — to\ < 5 (using now the fact that 7 G C 1 ). 

We apply to r) and <5 the chain rule in ||, Theorem 2.2.5, but in its viscosity rather than 
subgradient form (the viscosity form follows from the version given in that reference by applying 
the approximation theorem in Q, Proposition 3.4.5). This tells us that there exist t±, x, £, 
and (' such that |ti — to I < \ x ~ l(po)\ < <5, C ^ <k>V(x), and (' G <9d(C ■ 7)(*l)j so that 
|70l) - 7(*o)| < e and 1 77 - C'l < £■ 

Since 7 is differentiable, the scalar map ("-7 is too, and hence £' = (d/dt)((-j)(ti) = C'7'(*i)j 

so 

C = C-Y(tx) = C-(Y(*i)-Y(*o))+C-Y(to) 

= C-(Y(ti)-Y(to)) + ^yC-5(7(to)) 
= ^ + ^) C -^' 



s 



where 

= C • (Y(*i) - V(*o)) + • (<7(7(*o)) - , 



and therefore (' < 2e by fll2|), fll4|), and (|15|). So ?? < 3e. As e was arbitrary, we conclude r) < 0. 

■ 

Since > W(l) = V(£,) for all t < 1, for any /i > the last term in the following 

expression is nonpositive: 

^(V(a,0)-V(a-ah,0)) = l(yfr(l - h)) -V(a- ah,0)) + \{W(1) - W{1 - h)) . (16) 

Since V is locally Lipschitz, there is some constant L such that (evaluating 7(1 — h)): 

r\Vh(l-h))-V(a-ah,0)\ < La 3 h* (2 - hf/ 2 
h 

as h — > + . We conclude, by taking limits as h — > + in (pq) , that ^(0,0) < 0, as claimed. I 
3.3 Proof of Theorem [I] 



In order to be able to justify Remark 2.5 , we will prove Theorem |T| for a more general class of 



systems, namely those of the following general form: 

m 

x = g (x) + ^2 <P( u i)9i{x) (17) 

i=\ 

where (f(r) = \r\ p or (/?(r) = (signr) \r\ p (interpreting (f(0) = 0), and p is a fixed real number 
in the closed interval [1, 2]. Note that Theorem |l| as stated would correspond to p = 1 and the 
second choice of <p. We still suppose that states x evolve in ffi n , and inputs u — (uii • • • j^m) 
take values in IR m , and that the vector fields gi, % = 0, . . . , m are locally Lipschitz in x. 

The proof will be based upon the following general technical fact: 

Lemma 3.2 Assume that we are given: 

• an open subset O of R n ; 

• a continuous function a : O — ► M>o; 

• a continuous function [3 : O — > M>o; 

• an e > 0; 

• a continuous function V : O — ► M>o satisfying 

C • ^oN + J2 9i{x)J < -a(x) + (3{x) \u\ 2 VC G doVix) (18) 
for all x G and u £U. 
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Then, there exists a smooth W : O — > R such that 

|F(x) - W(a;)| < -V(i) (19) 

for all x G 0, and 

Vf(i) • ^o(^) + £ ft(a;)J < -a(x) + [(1 + e)0(x) + e] \u\ 2 (20) 

for all x G O and u £U. 

Before proving the lemma, we explain how to obtain Theorem [l] as a corollary. We pick a 
system S as in (Q), and any 7' > 7 > 0. We suppose given a continuous proper and positive 
definite V G W(S, 7), and need to show the existence of some W G W(S, 7') which is C 1 



on = R n \ {0}, in addition to being proper and positive definite. We pick, in Lemma 3.2 , 
/3(x) = 7, a(x) = \x\ 2 , and any e > such that (1 + 5)7 + e < 7'. The Lemma then applies to 
the restriction of V to O. We obtain a W as in the Lemma, and extend it from O to W 1 by 
defining W(0) := 0. The approximation property ( |l9| ) insures that is proper and positive 
definite, and it is continuous at zero as well. 



The proof of Lemma 3.2 will be based upon a reduction to the following result, which we 



quote from [15|: 



Lemma 3.3 Let E : x = f(x, d) be a system, with x G X = R n , and d G U, a compact metric 
space, so that f(x, d) is locally Lipschitz in x uniformly on d and jointly continuous in x and 
d. Assume that we are given: 

• an open subset O of X; 

• a continuous, nonnegative function V : O — > R satisfying 

C- f(x,d) < 0{x,d) Vx G O, C £ foV^x), d G W (21) 
with some continuous function : OxW->M; 

• two positive, continuous functions Ti and T2 on O. 

Then, there exists a smooth V : O — > R such that 

V(ar) - V"(x) < Ti(x) Vx G (22) 

and 

V7(i) • f{x,d) < G(x,d) + T 2 (x) VxGO, deli. (23) 

□ 

The proof given in [15] employs tools from nonsmooth analysis, borrowing in particular 
several simple facts from []|, || |l8| in order to pass from continuous V to locally Lipschitz 
V, followed by a standard smoothing argument as given in [16|. (To be precise, the result is 
stated and proved in using the language of proximal subgradients rather than viscosity 
subgradients. The proof, however, is the same in both cases.) 
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Proof of Lemma 3.2 

We define 



f(x,d) := (l - |d| 2 ) g (x) +£>(^)(l 



|2\ 2 



9i{x) 



i=l 



with IA = unit ball in R m . (If p = 2, we interpret the coefficient of g$ as simply <p(di).) Note 
that this system satisfies the hypotheses of Lemma 3.3. We use the same O and V, and let 

0(z, d) := - (l - M 2 ) a(x) + \d\ 2 0{x) . 

We need to verify that ( |2l| ) indeed holds. Pick any x and £, and any d with \d\ < 1. We treat 
separately the cases |d| = 1 and \d\ < 1. 

Case 1: \d\ = 1. If p < 2, then ( |2"T| ) holds trivially (because f(x,d) = and Q(x,d) = 
(5{x) > 0). So suppose p = 2. We must verify that 

m 

V>{di) C • 9i{x) < P(x) (24) 

i=l 

where y(r) = r 2 , or y(r) = r |r|, and d is a vector of unit norm. We are assuming that ( |l8|) 
holds, that is, 



C • g (x) + ^ y>(ui) C • 9i(x) < -a(x) + (5{x) \u\ 2 (25) 
i=i 

for all u. Now pick any u of the form (l/e)d, with e > 0, so e 2 tp(ui) = </?(dj) for each i. 
Multiplying both sides of (25) by e 2 , we have that 

m 



so taking limits as e —* one obtains the desired inequality 

Case 2: \d\ < 1. Introduce, for such a d, u = (ui, . . . ,u m ), where: 



di 



l-\d\ 2 



i = 1, . . . , m . 



Observe that 

2 



I |2 \ d \ i |j|2 1 | ,,2^i 1 f 

1 — \d\ 1 + \u\ v y 1 + \u 

for all i, so that 

f(x,d) 



- — 1 —~2 \9o(x) + V y(t**) flf«Cic) J 



and thus ( |lq ) implies that £ • /(x, d) < Q(x, d) for all £ E c^d^^)) as wanted. 
So, we apply Lemma |3.3| with 

Tx(x) := 
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and 

T2(x) := 5 min{l,a(x)} 
where 5 £ (0, 1) is picked such that 

< 1 + e and — — - < min <e, — ^ . 



l-<5 ~ 1-5 ~ 4 



This provides a function V. 

Now fix any u £ M m , and introduce 



<U := i = l,...,m. 

'l + \u\ 2 



Observe that the relations ( |26|) hold, and hence also 

|2 

"|2 



Ml 2 = r^<! 



1 + kt 



holds. Furthermore, go(x) + X/»=i f^i) 9i( x ) = (1 + M )f( x ^d). Therefore, ( f23| ) gives 



i=l 

< (1 + |u| 2 ) (- (l - |(f| 2 j a(x) + |d| 2 /?(x) + 5min{l, a(x)} 

= — a{x) + \u\ 2 (3{x) + <5min{l, a(x)} + |u| 2 <5min{l, a(x)} 

< —a[x) + |u| 2 (3(x) + <5a(x) + |it| 2 5 
= -(l-5)a{x) + \u\ 2 (f3(x) + 5) 



and if we now let 



we conclude that 



W := 



|V-W| < -r^—rV + 



1-5 1-5 



V -V 



4*- 



and 



( m \ 3(x) + 5 

9o{x) + 1 < ~a(g) + H < -a(x) + [(l + e)(3(x)+e}\u\ 

as desired for the conclusions of Lemma |3.2| . ] 
Notice, incidentally, that W is smooth not merely C . 



3.4 Results for systems of form (|8]) 

The positive claim in Remark 2J3, for p < 2, has been proved already. To show that Theorem |l] 
does not extend to systems (17) when p > 2, it will be convenient to introduce the vector field 
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Figure 1: The vector field g 



corresponding to what one might call an "L harmonic oscillator" . This is the locally Lipschitz 
vector field on M 2 defined by 

g(x u x 2 ) ■= ( '^'f 2 ) ■ (27) 
V - F2I x\ J 

The flow of g leaves invariant the L 1 -balls around the origin, see Figure |l[ Associated to g is 
the L 1 norm, seen as a proper, positive definite Lipschitz function 

Vi(x 1 ,x 2 ) := |xi| + \x 2 \ ■ 

Except for the fact that we do not now need the factor "2", this function is as before. So 
&dVi(x) = {(xi/\xi\,x 2 /\x2\)} when x\x 2 / 0, dvV\{x) = [-1, 1] x {x 2 /|x 2 |} when xi = and 
x 2 7^ 0, and similarly if x 2 = 0. Therefore, 

C • g = for all C e doVtix), x G M n . (28) 

We also introduce the following vector field: 

9o(xi,x 2 ) ■= ( ~\ Xl \ Xl ) . (29) 
\-\x 2 \x 2 J 

For the same Vi, 

(■g = -\x\ 2 for all ( G doV^x), x G M n . (30) 

Now fix any p > 2, and consider the following system S p , which is defined in terms of the 
above vector fields: 

x = g Q (x) + \ui\ p g(x) - \u 2 \ p g{x) (31) 

This is a system of type (|8|) with m = 2, g\ = g, and g 2 = —g. In view of ( |28| ) and (|30D, we 
have that 

C • (go(x) + l^i| p 5(^) - l«2| p 5(^)) < -\x\ 2 

for all x G M 2 and u = (u±, u 2 ) G M 2 , and every ^ G OdVi(x). This means that V\ G Woo(S p ,7), 
for any 7 > 0. We now see that the equality in Theorem [l] cannot hold. In fact, we prove the 
following far stronger statement: 

Proposition 3.4 For all V G Cq, and all 7 > 0, V G" W(E p ,i). 

Proof. Suppose that V G Cq and 

VF(i) • (go(x) + |-ui| p 5(x) - |n 2 | p 5f(2;)) < -\x\ 2 + 7 |m| 2 + 7 |n 2 | 2 (32) 
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for all x ^ and all u = (ui,u 2 ). Fix any x = £ ^ 0. Taking u = gives that VV(£) ^ 0. 
Since p > 2, letting U2 = and iti — > +oo gives VV(£) • g(£) < 0. On the other hand, u\ = 
and u 2 -> +oo gives - W(f ) • g(£) < 0. So we conclude that W(£) ■ g(£) = for all £ ^ 0. 

Now fix an a > and suppose that £ has the form (a, £2) with £2 7^ 0. Then, VV(£)'5(£) = 
means that 

(sign S2) ^(0,2:2) - V X2 (a,x 2 ) = 

Letting separately X2 — > + and X2 — > _ gives V^j (a, 0) = T4 2 (a, 0) = 3 contradicting 
W(a,0)/0. I 

A variation of this example is as follows. Fix again any p > 2. Instead of (pl|), we consider 
now the following system T,' p : 

x = go(x) + signti \u\ p g(x) (33) 

with m = 1. It is also true that Vi G yVoo(S p ,7), for any 7 > 0, and Proposition again 
holds, simply taking the separate limits as u — > +00 or u — > — 00 in its proof. 

Remark 3.5 There is nothing very special about the form (||). Mainly, we picked these systems 
in order to illustrate with a specific class when the theorem holds. But Theorem |l| holds also 
for a general class of systems of subquadratic growth in inputs. Specifically, we may consider 
systems of the general form x = g(x,u), where g is jointly continuous in x and u, and satisfies, 
for some constant p G (0, 2) and some continuous function h : R n — ► R: 

1. \g(x, it) I < h(x) (1 + \u\ p ) for every x and u; 

2. \g(x, u) — g(y, u)\ < c\x — y\ (1 + \u\ p ) for every u, and every x,y in some (arbitrary 
chosen) ball. 

The proof is the same as for systems of the form (||) . One only needs to set 

f(x,d) = (l-M^^dU-M 2 )- 1 ^ 

provided \d\ < 1, and f(x, d) = otherwise, and to replace everywhere go(x) + YliLi <-P{ u i)9i{ x ) 
by g(x,u). We omit the simple details. □ 

4 One-Dimensional Systems 

For one-dimensional systems, there is no gap between the locally Lipschitz and the Cq case: 

Proposition 4.1 Suppose given a system S as in (1), with n = 1. Assume that for some 7 > 
there exists a locally Lipschitz V G Woo (£,7). Then Woo (£,7) P| Cq 7^ 0. 



Proof. Pick a V as in the statement of the proposition. By Rademacher's theorem there exist 
a zero measure (Borelian) set JVcR and a continuous positive function h such that 

x N V'(x) exists and \V(x)\ < -h(x). (34) 
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We shall construct W only on M>o, the construction being similar on M<o- If x ^ N, dnV(x) = 
{V'(x)}, hence 

V'(x) (g Q (x) + u igi {x) \ < 7 \u\ 2 - x 2 V«G K. (35) 

Since the system x = go(x) is (globally) asymptotically stable, go(x) < for each x > 0. It 
follows from (||) and @) (with u = 0) that 



1 x 2 

-h(x) > V'(x) > , . r. > for x > 0, x iV. (36) 

2 |go(aOI 

Set for any x > 

F(x) := /p > : p ^#o0e) + ^iffiC^ol < 7 |^| 2 — x 2 Vm G Rj . 

We claim that the (closed convex) set F{x) is nonempty for any x > 0. Indeed, if x iV 
V'(x) G F(x). If x G iV we may pick a sequence (x n ) in M>o \ N such that x ra — » x as n — ► cxo. 
Since |V'(x n )| < ^h(x n ) < c for some constant c, we may extract a subsequence of (V'(x n )) 
which converges towards some p > 0. Clearly p G F(x). 

Let us set 

m 

a := ^2 gi{xf , 6:=475f (x), c := 47X 2 
i=l 

and 

A(p) := ap 2 + 6p + a 
It is a straightforward exercise to see that for any p > 0, 

p G F(x) A(p) < 0. (37) 

By the claim, A(p) < for some p, whereas A(p) — > cxd as p — > — oo (since 6 < 0). It follows 
that A has (at least) one real root, hence ft 2 — 4ac > 0. Set 

f h[x) if a = 0, 

:= { min [h(x), if a + . (38) 

Note that —6 = 47 |<7o( a; )| > on x > 0, so when a — > the second expression in the minimum 
above becomes unbounded; from here, it follows that p is continuous on M>o- We claim that 

V'{x) < p(x) Vx € R> \ N. (39) 



Pick any x G R> \ -/V. If a = 0, then fl39j) follows from fl36|) and fl38|). If a / 0, since 
F'(x) G F(x), we see that A(V r/ (x)) < (by (|37|)), hence V'(x) < ~ fc+V 2 fc a "~ 4ac which, combined 
with (|D and (||), yields V'(x) < p(x). 

We are now ready to define W on R>o: we set W(x) = Jq p(s) ds for any x > 0. Since 
< p < h, W is a (well-defined) locally Lipschitz function on R>q which is C 1 away from 0. 
Integrating in ( p9| ) we get W(x) > V(x) > for any x > and linr^oo W(x) = cxd. Finally we 
claim that 

A(p(x)) < Vx G M> \ N. (40) 
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Pick any x G R >0 \N. If a = 0, then 

A(p(x)) < A(y'(x)) (using b < and (| 
<0 (by©). 

If a 7^ 0, we are led to prove that 

, , ~-b-Vb 2 -4ac -b + yjb 2 - Aac 
P{X) G [ 2a ' 2a 

Owing to the definition of p(x), we only have to prove 



7 / v _ -b - Vb 2 - 4ac 
HX) * 2a • 

But 

-b - Vfr 2 - 4ac _ 2c 2c _ 2x 2 

2^ ~ \b\ + V& 2 -4ac ~ H ~ isoR] ~ 

by (36). This completes the proof of (f40|). Note that (pC|) also holds true for any x > 0, by 
continuity of A op. Using (^) it means that W'(x) = p(x) G F(x) for all x > 0, i.e. 

W'(x) \ m(x) + J2 u i9i(xYj < 7 \u\ 2 -x 2 VnG M. 

The proof of the proposition is complete. I 

We now show that, for systems not affine in inputs, there is again a gap between the 
Lipschitz and differentiable cases. In Proposition |2.2| we gave an example of a system Si and a 
Lipschitz Vi £ Woo (Si, 1), such that no V in W(£ 1; 1) f| c o 

can be positive definite or proper. 
The system was two-dimensional (n = 2) and had two-dimensional input (m = 2). We now 
provide a scalar (n = m = 1) system E3 which has the following properties: (a) it admits a 
Lipschitz V3 G Woo(£3, 1), but (b) any W in W(E3, 1) must be non-differentiable. Thus, the 
conclusions are even stronger than for Si; on the other hand, S3 is not affine in inputs. 

Before giving the form of S3, we start by considering the following two functions ip+ and 
: R 2 -> R: 

ip+(s,t) := max{min{(s — t)/2, s} , 0} 

and 

ip-(s,t) := — max{min{(— s — t)/2, — s} , 0} . 
Both of these are Lipschitz, and <p+(0,t) = (p-(0,t) = for all t. Thus, the function 

(p(s,t) := (sign s) max |min I -(|s| — t), |s| I , 

(with (p(0, t) = 0) obtained by using ip + for s > and cp- for s < is also Lipschitz. 
As min {^(|s| — t), |s|} < \s\, it follows that 

\<p(s,t)\ G [0,|a|] 
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for all (s, t). Since sign <^(s, t) = sign s, this means that ^(s) £ [0, s] when s > and (/?(s) S [s, 0] 
when s < 0. When i > |s|, min — t), |s|} = |(|s| — t), so 

<p(s,i) = if t > \s\ . 

If t < - \s\ then |s| = |s| /2 + |s| /2 < |s| /2-t/2, so min{|(|s| - t), |s|} = |s|. Therefore 

c/?(s, i) = s if t < — |s| . 

The diagram in Figure || summarizes this information about the range of tp. 

'S = t 




Figure 2: The range of if 



We define now, for a, b > 0: 

ip(a,b) :-- 



Lp(b - a,b + a - 2) + (b - a) 



As \b — a\ < 6 + a — 2 if and only if both a > 1 and 6 > 1, VK ; b) = ^(b — a) for such a, b. 
Similarly, the properties of ip also imply that ip(a,b) = b — a when a < 1 and 6 < 1, that 
^(ct, 6) E [6 — a, | (6 — a)] when a > b, and that ^)(a, 6) G [| (6 — a), 6 — a] when a < b. We are 
now ready to specify S3. We let: 



for x > 0, and 



f(x,u) := (\u\ +x)i/j(x,\u\) 



f(x,u) := x 2 + \u\ ip(0, \u\) 



for x < 0. This function is locally Lipschitz. Notice that ^(0, \u\) > \u\ /2 > for all u, so that 
f(x,u) > x 2 for x < 0. Observe the following properties, for all x > 0: 



1. x < 1 & H < 1 

2. x < 1 & H > 1 

3. x > 1 & |u| < 1 

4. x > 1 k \u\ > 1 



/(x, u) = u 2 — x 2 . 
/(x, n) < ti 2 — x 2 . 
/(x,u) < |(u 2 -x 2 ). 
/(x,u) 



±(u 2 



Finally, the Lipschitz function V3 shown in Figure || 

We show that V3 G Woo (S3, !)> that is, y'(x)/(x, u) < u 2 — x 2 for all x 7^ and all u. For 
x < this is obvious, since V (x)/(x, = —f(x, u) < —x 2 < u 2 — x 2 for all u. So we only need 



to analyze the case x > 0. For any < x < 1, V'(x)f(x,u) = f(x,u) < u 



(properties [j] 



and |2|) while for x > 1 we have V'(x)f(x, u) = 2/(x, u) < u 2 — x 2 as well (properties || and Q). 
Finally, we deal with the nondifferentiability point x = 1. An easy calculation shows that 
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Figure 3: V%{x) := max{|x| ,2x — 1} 



cfoV^l) is the closed interval [1,2]. Pick any p £ [1,2]. When \u\ < 1, f(x,u) = u — x 2 < 0, 
so p > 1 implies pf(x,u) = p(u 2 — x 2 ) <u 2 — x 2 . Finally, if \u\ > 1, f(x,u) = \(u 2 — x 2 ) > 0, 
then p < 2 implies pf(x, u) < u 2 — x 2 as well. 

We now show that any W in W(£3, 1) must be non-differentiable. Suppose that there is 
some such W which is differentiable. Fix u = 1. For any x, we must have, since W G W(T,^, 1): 



W"(x)/(s,l) < 1 



x 



2 



When x £ (0, 1), f(x, 1) = 1 — x 2 > 0, so W(x) < 1. On the other hand, for x > 1 we have 
f(x, 1) = (1 — x 2 )/2 < 0, so W'{x) > 2 for such x. If W £ Cq, this gives a contradiction as 
x — > 1 + and x — ^ 1~. However, continuity of W' is not needed for the contradiction, since we 
can argue as follows. By the mean value theorem, 

W{x)-W(l).. . ,W(x)-W(l) 
hmsup — — — < 1 < 2 < hmmf — — — , 

x -+\- x — 1 x^l+ x — 1 

which contradicts the existence of W'(l). I 
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